Learning City Structures from Online Maps

Huge amounts of remote sensing data are nowadays publicly available with applications in a wide range of areas including the automated generation of maps, change detection in biodiversity, monitoring climate change and disaster relief. On the other hand, deep learning with multi-layer neural networks, which is capable of learning complex patterns from huge datasets, has advance greatly over the last few years. This work presents a method that uses publicly available remote sensing data to generate large and diverse new ground truth datasets, which can be used to train neural networks for the pixel-wise, semantic segmentation of aerial images. First, new ground truth datasets for three different cities were generated consisting of very-high resolution (VHR) aerial images with ground sampling distance on the order of centimeters and corresponding pixel-wise object labels. Both, VHR aerial images and object labels are publicly available and were downloaded from online map services over the internet. Second, the three newly generated ground truth datasets were used to learn the semantic segmentation of aerial image by using fully convolutional networks (FCNs), which have been introduced recently for accurate pixel-dense semantic segmentation tasks. Third, two modifications of the base FCN architecture were found that yielded performance improvements. Fourth, an FCN model was trained on huge and diverse ground truth data of the three cities simultaneously and achieved good semantic segmentations of aerial images of a geographic region that has not been used for training. This work shows that using publicly available remote sensing data can be used to generate new ground truth datasets that can be used to effectively train neural networks for the semantic segmentation of aerial images. Moreover, the method presented here allows to generate huge and in particular diverse ground truth datasets that enable neural networks to generalize their predictions to geographic regions that have not been used for training.